Comparing Bibliometric Statistics Obtained from the Web of Science and Scopus

For more than 40 years, the Institute for Scientific Information (ISI, now part of Thomson Reuters) produced the only available bibliographic databases from which bibliometricians could compile largescale bibliometric indicators. ISI’s citation indexes, now regrouped under the Web of Science (WoS), were the major sources of bibliometric data until 2004, when Scopus was launched by the publisher Reed Elsevier. For those who perform bibliometric analyses and comparisons of countries or institutions, the existence of these two major databases raises the important question of the comparability and stability of statistics obtained from different data sources. This paper uses macrolevel bibliometric indicators to compare results obtained from the WoS and Scopus. It shows that the correlations between the measures obtained with both databases for the number of papers and the number of citations received by countries, as well as for their ranks, are extremely high (R ≈ .99). There is also a very high correlation when countries’ papers are broken down by field. The paper thus provides evidence that indicators of scientific production and citations at the country level are stable and largely independent of the database. Background and research question For more than 40 years, the Institute for Scientific Information (ISI, now part of Thomson Reuters), produced the only available bibliographic databases from which bibliometricians could compile data on a large scale and produce statistics based on bibliometric indicators. Though often criticized by bibliometricians (see, among several others, van Leeuwen et al. 2001 and Moed, 2002), Thomson’s databases—the Science Citation Index (Expanded), the Social Sciences Citation Index and the Arts and Humanities Citation Index, now regrouped under the Web of Science (WoS)—were the major sources of bibliometric data until 2004, when Scopus was launched by the publisher Reed Elsevier. For those who perform bibliometric analyses and comparisons of countries or institutions, the existence of these two major databases raises the important question of the comparability and stability of statistics obtained from these two different data sources. The comparison of these two databases has been the focus of several papers, mostly made using the “bibliographic” web versions of the databases. For instance, Burnham (2006), Bosman et al. (2006), Falagas et al. (2008), Gavel and Iselid (2008), Jacsó (2005), Neuhaus and Daniel (2008) and Norris and Oppenheim (2007) compared the general characteristics and coverage of the databases; other studies compared the bibliometric rankings obtained. Given the limitations of the databases’ web versions for producing bibliometric indicators, most of these studies used small samples of papers or researchers. For instance, Bar-Ilan (2008), Belew (2005), Meho and Yang (2007), Meho and Rogers (2008) and Vaughan and Shaw (2008) compared small samples of researchers’ citation rates and hindexes. Along the same line, Bakkalbasi et al. (2006) and Lopez-Illescas, Moya-Anegon & Moed (2008; 2009) compared citations received by a sample of journals in oncology. One of the few macro-level bibliometric studies is that of Ball and Tunger (2006), which compared the citation rates obtained with the two databases. These studies generally found good agreement between the WoS and Scopus, which is not surprising given the fact that 7,434 journals—54% of Scopus and 84% of the WoS—are indexed by both databases (Gavel and Iselid, 2008). However, they do not show whether the differences in article citation rates observed between the two databases affect the rankings of countries or institutions. Whereas the previous papers mainly used the online version of these databases, this paper is written by licensees of these tools and is therefore based on bibliometric production platforms (implemented on Microsoft SQL Server). Using these platforms, the paper compares macro-level bibliometric indicators and provides a comparative analysis of the ranking of countries in terms of the number of papers and the number of citations received, for science as a whole as well as by fields in the natural sciences and engineering. The convergence of the bibliometric indicators will suggest that 1) the two databases are robust tools for measuring science at the country level and that 2) the dynamics of knowledge production at the country level can be measured using bibliometrics. Using these data, the present paper, which builds on a previous abstract presented at the STI2008 conference in Vienna (Archambault, Campbell, Gingras and Larivière, 2008), examines how countries’ rankings compare for both the number of papers and the number of citations. In addition to these correlation analyses based on rankings, the number of papers and the number of citations obtained in both databases at the country level are also examined. The paper then goes one step further by examining how comparable scientific output at the country level in scientific fields such as physics, chemistry and biology is. Finally, the paper examines output at the country level in the field of nanotechnology. 1 More often than not, these studies also included Google Scholar. Given that this database is not yet suitable for compiling macro-level bibliometric data, this paper compares only Scopus and the Web of Science.